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METHOD FOR DETECTING A PERSON IN A SPACE 
Background Information 

The present invention is directed to a method for detecting a 
person in a space according to the species defined in the 
independent claim. 

Advantages of the Invention 

The method according to the present invention for detecting a 
person in a space having the features of the independent claim 
has the advantage of being a compromise between complexity and 
simplicity: Complex situations, such as persons in unusual 
poses, with unusual clothes, or persons that are partially 
concealed, are processed with little effort. In this 
connection, it should be noted that the "detection of a 
person" is understood to also include the detection of 
specific body parts. 

During the search, the presented method concentrates on 
comparatively few body parts that can be detected in a 
reliable and fast manner, such as the head. If the head of an 
occupant is in an unusual position in the motor vehicle, or if 
there are two occupants per seat in the motor vehicle, or if 
there exist partial concealments of the body, the respective 
heads are still found because there is no need to bring a 
complete human model into line with the scene. 

The method provides different sub-models for a body part, the 
different sub-models being a function of the method in the 
case of different clothing and headgear. 



Therefore, the method of the present invention is superior 
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both to the model -free method and to a complex, complete CAD 
model because it can also cope with atypical scenarios. 
"Model -free" means that here the volume distribution, or other 
characteristics such as the curvature or texture, are directly 
5 evaluated by a classifier. A serious disadvantage of this 
method is that not all situations (such as the pose or 
clothing) can be covered in the training. This results in poor 
detection performance for model -free approaches. 

10 Unlike the CAD model, the method according to the present 

invention is fast because only a few parameters are estimated. 
For a fast algorithm, it is also advantageous that not all 
measured values, but only those of the volume to be examined 
have to be taken into account to adapt a sub - mode 1 . The method 

15 according to the present invention has the characteristic that 
a hierarchical model of a human is adapted to the spatial data 
of a depth sensor. This depth sensor preferably has high 
resolution. In particular, the organization of the model and 
the method of adaptation to the spatial data of the depth 

20 sensor are advantageous. In this context, the model is a shape 
model (surface model) of the human body. The model is designed 
to be articulatable, i.e., movable, and to be able to describe 
different forms of bodies and clothing. 

25 Advantageous improvements of the method for detecting a person 
in a space described in the independent claim are made 
possible by the measures and further refinements specified in 
the dependent claims . 

3 0 It is particularly advantageous that the complete model is 

adapted to follow the person over time by further adapting the 
sub-models using the data at predetermined intervals. This 
allows a so-called "tracking"; a complete adaptation no longer 
being needed, but the detected body parts being trackable in 

3 5 space and time. 



It is also advantageous that at least one body part, such as 
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the head, the shoulder, or also, for example, the chest of a 
human, is of particular importance for the classification of 
an occupant, especially in automotive applications such as 
restraint systems or occupant condition monitoring. Moreover, 
5 these body parts are particularly easy to recognize. 

Moreover, it is beneficial that intensity information from the 
data is used, the intensity information being texture 
characteristics, such as gray levels of the gray image or 

10 color values of the color image, which are each determined by 
an imaging sensor. These texture characteristics may then be 
associated with individual body parts in order to associate 
more detailed information with the individual body parts. This 
also allows consistency checking to be able to detect a 

15 person. 

However, the general consistency check is performed using the 
position parameters that define the relationship between the 
individual sub-models associated with the different body 
20 parts. This consistency check allows the adaptation of the 

sub-models by the measured sensor values to lead to reasonable 
values changes. 

Moreover, it is an advantage that the complete model is 
25 transmitted for occupant classification to a restraint system 
in a vehicle in which the person is located. This then allows, 
in particular, detection of persons; the system primarily 
detecting the posture of the occupant to identify a dangerous 
position of the occupant with respect to air bags and belt 
30 pretensioners . Thus, the so-called n out-of -posit ion positions" 
are identified to prevent activation of the air bags in such 
positions in order to avoid injuries. The so-called "tracking 11 
allows dynamic out-of -position sensing. Thus, the posture of 
the occupant is detected in real time. This plays a role 
35 especially in the case of fast movements of the head relative 
to the sensor, which are induced, for example, by vehicle 
decelerations during braking maneuvers or collisions. Here, it 
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is also possible to classify the occupants based on size or 
biometric conditions, such as the average head radius or the 
shoulder width. 

5 Another use of the method according to the present invention 
is the anti-pinch protection for extremities or objects in 
power windows. Extremity detection can also be used for 
gesture recognition. Monitoring the passenger compartment 
while the vehicle is parked can be an application of the 

10 method according to the present invention. This may be used, 
in particular, in a car alarm system. It is also possible to 
automatically operate the child security door lock against 
accidental opening if a child has been detected by the method 
according to the present invention. In combination with a 

15 temperature sensor, it is possible to adjust the temperature 
by opening and closing the windows, or by controlling the air 
conditioning, in case a person or, in particular, a child is 
alone in the vehicle. Convenience features can also benefit 
from the method according to the present invention. The 

2 0 adjustment of headrests, mirrors, sun visors, steering wheel 
and/or contoured seats can be carried out automatically. The 
control of other systems that require the exact position of 
the head is rendered possible, inter alia, a hands-free kit, 
head-up displays, or noise control systems. Other methods, for 

25 example, for identifying the occupants, or eye localization 

are aided by a precise localization of the head, if necessary, 
along with an orientation measurement. 

The depth sensor takes the form, in particular, of an image 
30 pickup. Above all, a video sensor, such as a stereo camera 

system or a multi-camera system, may be used here. Also usable 
here are a distance -measuring matrix sensor using the 
transit -time principle or the phase -measuring principle, a 
laser scanner, a laser radar, or structured lighting. Further, 
35 the distance-measuring matrix sensor may use a microwave 

radar, or the measuring principle may be based on the emission 
of sound waves (ultrasound) , or an arbitrary electromagnetic 
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radiation (visible light, invisible light, microwaves). 

Likewise, the representation of the sub-models contained in 
the modules can take completely different forms, such as CAD 
5 data, measurement data, triangulated data, superquadrics , or 
two-dimensional descriptions . 

Drawing 

10 Exemplary embodiments of the present invention are depicted in 
the drawing and will be explained in greater detail in the 
following description. In the drawing, 

Figure 1 shows a block diagram of a device according to the 
15 present invention; 

Figure 2 is a first flow chart of the method according to the 
present invention; 

2 0 Figure 3 shows a block diagram of the method according to the 

present invent ion ; 

Figure 4 is a representation of the hierarchical models; 

25 Figure 5 shows a shape model of the head in different 

variations and depths of detail; 

Figure 6 is a second flow chart of the method according to 
the present invention; and 

30 

Figure 7 shows different mounting positions to cover the rear 
passenger compartment . 

Description 

35 

Systems for sensing the passenger compartment of a motor 
vehicle are based on very different physical measuring 
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principles. For example, it is possible to use a weight mat 
here; the pressure profile being used to determine whether the 
seat is empty or occupied. In further refinements, it is 
possible to carry out a classification. However, the sensor 
5 systems relevant to the method according to the present 
invention are depth-measuring sensor systems. Here, the 
distance-measuring matrix sensor preferably takes the form of 
an image pickup. Such an image pickup may be a video sensor, a 
radar sensor, an ultrasound sensor, or a multi-camera system. 
10 In this connection, it is possible to use different measuring 
principles: the transit-time principle, the phase -measuring 
principle, the principle of structured lighting, and the laser 
scanner principle. 

15 The spatial data of the depth sensor, which generally takes 
the form of a matrix sensor and produces more than a hundred 
measured values virtually simultaneously, is used to detect a 
person in a space. In the case of the passenger compartment of 
a motor vehicle this is, in particular, the occupant -presence 

20 situation. Besides the pure occupant presence, it is also 
necessary to classify the occupants, i.e., size, pose, and 
weight. In this connection, especially the localization of 
individual body parts, such as the head or the chest, is of 
great interest for the control of, for example, restraint 

25 systems. Such information is also useful for various 

convenience features, such as the adjustment of rearview 
mirrors . 

Figure 1 shows a device according to the present invention in 
30 a block diagram. A person 6 is detected by a depth sensor 1 in 
terms of his/her spatial extent. Depth sensor 1 is mounted, 
for example, in a passenger compartment to act as an image 
pickup. The spatial data are transmitted to a processor 2 on 
which the method according to the present invention is 
35 executed. A restraint system 3, an ant i -pinch function 4, and 
convenience features 5 are controlled by processor 2 as a 
function of the result of the method according to the present 
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invention. Therefore, processor 2 may be located in a control 
unit . 

Further vehicle functions are also controllable by processor 2 
5 so that these vehicle functions may also benefit from the 

inventive method for detecting a person. However, the method 
according to the present invention is also of great interest 
outside of motor vehicles, for example, for video 
surveillance. Access systems may also benefit from the method 
10 according to the present invention. 

Figure 2 shows the sequence of the inventive method in a first 
flow chart. In method step 6, sensor 1 produces sensor values 
that provide spatial data about person 6. This data is 

15 transmitted to processor 2 . Processor 2 loads on its memory a 
model of a human which is hierarchically structured; i.e., 
there exist sub-models for different body parts, the 
sub-models in turn being divided into further sub-models. 
This subdivision is done to allow optimal adaptation for 

20 different typical clothing situations of the particular body. 
Then, processor 2 selects the appropriate sub-model at the 
lowest level of the sub-models, and performs an adaptation 
using the spatial data. For this purpose, any optimization 
method may be employed. 

25 

Method step 8 checks whether this adaptation is consistent 
with respect to the position parameters. If this not the case, 
then the method returns to step 7 to repeat the adaptation. 
However, if this is the case, then, in method step 9, the 

30 complete model is gradually assembled from the sub-models. If, 
in the process, certain body parts are not found, for example, 
because of a concealment effect, a consistent interpretation 
of the spatial data is still obtained. This complete model is 
then used to recognize the person. This data is then made 

35 available to other functions, such as the restraint system, 
the convenience feature, or the ant i -pinch protection. 
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Figure 3 explains the inventive method in a block diagram. A 
person 11 is seated on a vehicle seat 10, and is spatially 
detected by a depth sensor 12 . This depth sensor 12 takes the 
form of a video sensor here, and is located above the 
5 windshield. However, depth sensor 12 can also be installed at 
other locations in the passenger compartment. The spatial 
measured values of sensor 12 are transmitted to a processor 13 
which adapts the spatial . model to the spatial measured values 
using the sub-models. To this end, a complete model 14 is 
10 loaded from a memory. Processor 13 delivers the size, 

position, and posture of the occupant as an output quantity 
15 . 

In Figure 4, the hierarchical model is explained by way of an 

15 example. The complete model is referred to as "shape model 0 
16". This model 16 is further divided into one or more units 
17 through 19, which are referred to as shape model 1 through 
shape model 3 . The permitted relationships between these 
sub-models, which are defined by the relative position 

2 0 parameters, are stored for these sub-models. Analogously to 
the division of shape model 0 into its sub-models, it is 
possible to break down these sub-models into sub-models as 
well, and to specify the relationships of the sub-models. The 
sub-models of shape model x are referred to as "shape model 

25 x.y". In this connection, y denotes a non-zero natural number. 
The model may contain an arbitrary number of such levels. The 
number of the further subdivisions and the depth of the 
subdivisions, i.e., the number of levels, are not 
predetermined and may be handled differently for different 

30 sub-models. All branches of the model have in common that they 
contain one or more surface models as the last level. The set 
of surface models describes the respective body part in 
different forms and depths of detail. The shape model 1.17 
sub-model is composed of sub-models for head 27. Shape model 

35 2.18 is divided into shape models 2.1.20 and 2.2.21 of which 

shape model 2.1 pertains to upper arm 2 8 while shape model 2.2 
pertains to lower arm 29. Shape model 3.19 is divided into 
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three shape models 3.1.22, 3.2.23 and 3.3.24, which are 
defined by the relative position parameters with respect to 
each other. Shape model 3.1.22 is divided into shape models 
3.1.1 and 3.1.2, which are provided with reference numerals 25 
5 and 26, respectively. Shape model 3.1.1 is divided into 

further sub-models 30 that define the shoulder here. Shape 
model 3.1.2 defines different pieces of torso information. 
Likewise, body parts 32 and 33 are associated with shape 
models 3.2.23 and 3.3.24, respectively. The number of the 

10 further subdivisions and the depth of the subdivisions, i.e., 
the number of levels, are not predetermined and may be handled 
differently for different sub-models. All branches of the 
model have in common that they contain one or more surface 
models as the last level. The set of surface models describes 

15 the respective body part in different forms and depths of 

detail. During the adaptation of the model to the scene, the 
model is processed starting from certain shape models or 
individual surface models. These individual parts are 
assembled to form the complete model, taking into account the 

20 consistency with respect to the position parameters. Once the 
rough position parameters have been determined, detailed 
surfaces may be further adapted, taking into account the 
consistency conditions. 

25 If certain body parts are not detected, for example, because 
of concealment effects, a consistent interpretation of the 
spatial measured data is still obtained. 

An exemplary set of surface models of the head is represented 
3 0 in Figure 5. The complexity of the model variants increases on 
abscissa 34. An increasing depth of detail of the model is 
described on ordinate 35. The simplest model 36 becomes the 
more complex sub-model 37, which is sub-divided into a more 
detailed model 38 and an even more detailed model 40; the more 
35 detailed model 40 already representing the head shape fairly 
accurately. For model 38, there exist model 3 9 which has the 
same depth of detail, but greater complexity. Selecting a 
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greater depth of detail for model 3 9 leads to model 41, which 
shows a head with a hat that has a brim. Sub-models 40 and 41 
are ultimately used for adaptation to the spatial data. 



5 Figure 6 is a second flow chart of the method according to the 
present invention. In method step 43, depth sensor 1 produces 
the spatial data. Method step 45 defines a region to be 
examined. This is done using data of already processed shape 
models: the detection status, position, and position 
10 parameters from method step 44. In method steps 46 and 47, the 
model is adapted to the spatial data, taking into account the 
spatial shape model from method step 42 . This step may be 
carried out, for example, using a standard method of model 
adaptation in terms of the minimum square error. 

15 

If this adaptation was successful, then, in method step 48, 
the detection status is stored in terms of position and 
position parameters. In method step 49, the position and 
position parameters of the model are then checked for 

20 consistency. To this end, the data of already processed shape 
models from method step 44 is reused. In method step 50, the 
results of the adaptation are recorded in the detection status 
of the sub-model. The detection status and the position 
parameters are passed on to the next higher level. If, again, 

25 relationships exist with another model, then a consistency 
check is performed until the complete model has been 
assembled. Once the model has been adapted, a subsequent, 
complete adaptation is no longer needed, and the detected body 
parts can be tracked in space and time. For fast and stable 

3 0 implementation, only body parts that can be detected quickly 
and reliably should be included in the model, typically the 
head, torso, and arms. The head module and the upper part of 
the body play a central role in the model because the human 
head is of outstanding importance to the set objective and, in 

35 addition, is comparatively invariant in shape. Moreover, the 
head is usually spatially exposed, and can therefore be 
located relatively reliably. The presented method is generally 
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independent of the source of the spatial data. 

Figure 7, image (a) , shows an additional sensor for covering 
the rear passenger compartment. Figure 7b, in contrast, shows 
an alternative mounting position of the sensor for covering 
the entire rear passenger compartment. This is denoted by- 
reference numeral 52, whereas in Figure 7a, sensor 53 is 
provided for the front seat and sensor 51 for the rear seat. 
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